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ABSTRACT 



The New England Head Start Teaching Center (NEHSTC) is one 
of 14 federally funded programs created to test the efficacy of 
participatory, hands-on training for enhancing Head Start service delivery. 
An outcome evaluation of the program was conducted after 3 years of 
operation. The research design of the evaluation was a nonequivalent 
comparison group design with information collected prior to training, and 1 
and 6 months after training. Comparison subjects were drawn from Head Start 
agencies in the New England region who had not participated in NEHSTC 
training. Data were collected through surveys and on-site observations of 
randomly selected trainees 6 months after training. A total of 239 trainees 
comprised the NEHSTC group and 130 the comparison group. Training was 
provided to Head Start staff in groups of up to 12 persons during a 3- to 
5-day period of residence at the Teaching Center and included areas such as 
cultural diversity, individualizing services, home visiting, food safety, 
director development, and child- initiated activities. Four types of learning 
activities were used: didactic, group discussion, simulation/role play, and 
participatory. The findings demonstrated that the NEHSTC was successful in 
implementing high quality, participatory training within the context of an 
on-going Head Start program. The outcome findings indicated that staff who 
participated in the NEHSTC training demonstrated gains in knowledge, skills, 
and attitudes compared to similar Head Start employees who did not receive 
such training. The positive findings suggest that participatory training 
should be included in the menu of training options available to Head Start. 
(Author/KB) 
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Abstract 

The design and evaluation of the New England Head Start Teaching Center (NEHSTC), one of 14 
federally funded programs created to test the efficacy of participatory, hands-on training for enhancing 
Head Start service delivery, is the focus of this paper. The unique characteristics of the NEHSTC and the 
outcome evaluation results from 3 years of training will be presented. The findings demonstrate the 
NEHSTC was successful in implementing high quality, participatory training within the context of an on- 
going Head Start program. Outcome findings indicate staff who participated in the NEHSTC trainings 
demonstrated gains in knowledge, skills, and attitudes compared to similar Head Start employees who did 
not receive training. The positive findings suggest that participatory training should be included in the 
menu of training options available. Because of the unique size and scope of Head Start, the success of 
ongoing efforts to improve the quality of its programs and services are particularly significant. Within 



Head Start, this discussion of quality enhancements via innovative training models is timely given the 



advent of the new performance standards and the restructuring of the Training and Technical Assistance 



system. Additionally, the findings are relevant for broader efforts to improve early care and education 
programs nationwide. 
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Introduction 

The Head Start Teaching Center was a federally funded national demonstration project designed 
to test the efficacy of participatory training for staff in all components of Head Start services. When the 5 
year project was initiated by the Administration for Children, Youth, and Families (ACYF) of the U.S. 
Department of Health and Human Services in 1992, 14 Head Start Teaching Centers were funded 
including at least one in each Head Start region. As required by ACYF, each Teaching Center project 
included an independent evaluation to ascertain the effectiveness of training. 

The 14 Head Start Teaching Centers were individual Head Start programs deemed by ACYF as 
providing exemplary Head Start services in all component areas and as capable of providing quality 
training to visiting Head Start staff. Head Start Teaching Centers were expected to provide training that 
was “participatory in nature, relying on observation, guided practice and immediate feedback" (Federal 
Register, 1992) in each of the four major Head Start components: education, health, social services, and 
parent involvement. The Teaching Center concept was based on the teaching hospital model in medicine 
that provides quality medical services to patients and training to visiting doctors. 

The 14 Head Start Teaching Centers funded by ACYF represented diverse approaches to 
implementing the Teaching Center model. Approaches ranged from the training of trainers to training of 
individual Head Start direct service or administrative staff. While some programs provided for training a 
relatively small number of persons with ongoing follow-up activities over an extended period, others 
consisted of 3 to 5 day training sessions without any follow-up training and therefore had the potential to 
provide training for several hundred of trainees in a given year. Evaluation data collected over the 5 year 
demonstration project should add considerably to our knowledge of the efficacy of these different 
approaches to participatory training in early childhood education. 

The New England Head Start Teaching Center 

The New England Head Start Teaching Center (NEHSTC), designed and implemented by CHILD, 
Inc., a Head Start grantee located in Rhode Island, provided intensive training during a relatively short (3 
to 5 day) period of residence at the Teaching Center. The model permitted a large number of Head Start 
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staff from New England to be trained in a variety of relevant topics including, cultural diversity, 
individualizing services for teachers and family workers, home visitor training, home visiting program 
development for Head Start directors, food safety and sanitation, menu planning, and child initiated 
activities. Although the NEHSTC served a large number of trainees in a given year, the nature of the 
training design ensured that individual needs were accommodated. The overall participatory model used 
at the NEHSTC consisted of four types of learning activities: didactic, group discussion, simulation/role 
play, and participatory. The rationale for using didactic lessons, discussion, and simulation activities at 
the NEHSTC was to provide foundation information necessary to make the participatory training most 
meaningful, rather than as stand alone information delivery approaches typical of conference training 
formats. Additionally, enrollment for each training session was limited to no more than 12 participants. 
This format allowed for ample discussion, guided practice, individualized feedback, and participant 
sharing. This format also enabled trainees to play an active role in shaping the training to meet their own 
needs. Although a training syllabus was prepared by the trainer, the nature of the model encouraged 
trainees to initiate discussion of their own concerns and thus individualize the training. 

Training at the NEHSTC was conducted by the Teaching Center Project Coordinator, a staff 
member hired to coordinate the project, and other component coordinators (e.g., Education, Parent 
Involvement, Health, and Social Services) as well as the Executive Director. Other Head Start staff 
assisted with the training by serving as “training partners" during participatory sessions. Training partners 
were direct service staff (e.g. classroom teachers, home visitors, cooks) of the Head Start Teaching 
Center who facilitated the observational and guided practice training activities. 

Evaluation Plan 

The evaluation plan for the NEHSTC consisted of two major components: 

1. a formative evaluation of the development of the NEHSTC during the planning year (1992-93). 
The purpose was to examine the process of developing the teaching center including management 
issues, design of training strategies and topics, preparation of staff for training roles, and evaluation of 
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pilot training. The results of the formative evaluation are reported in Caruso, Horm-Wingerd, and 
Dickinson, 1996. 
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2. an outcome evaluation to assess the impact of training during the implementation years of 
NEHSTC training (1993-97). The outcome evaluation used a nonequivalent comparison group design 
with information collected prior to training, and 1 and 6 months after training. Comparison subjects were 
drawn from Head Start agencies in the New England region that had not participated in NEHSTC training. 
Training and comparison subjects and their supervisors provided information via mailed surveys that were 
specifically designed to assess knowledge, skills, and competencies in the training content areas. On-site 
observations of randomly selected trainees occurred 6 months after training as an independent 
assessment of the impact of training (see Table 1). 

Results 

Demographics 

Subjects. During the 1993-94, 94-95, and 95-96 training years, 239 trainees participated in the 
NEHSTC training and 130 individuals participated in the comparison group "treatment," a stress 
reduction or conflict resolution workshop. Of the 239 potential subjects in the training group, 136 
(56.90%) supplied pretraining and 1 month posttraining data and 91 (38.08%) supplied pretraining, 1 
month, and 6 month posttraining data. Of the 130 potential comparisons, 97 (74.62%) supplied 
pretraining and 1 month posttraining data and 41 (31.54%) supplied pretraining, 1 month, and 6 month 
posttraining data. In general, the subjects for the study were white females between the ages of 25 and 
45. The typical subject was married with an average of 2 children and an annual family income of 

$20,000 to $30,000. The majority had some college or an associates degree and had worked in Head 
Start from 3 to 5 years. 

Analyses indicated that no significant differences existed between individuals (both trainee and 
comparison subjects) who did and did not complete all portions of the evaluation component on the seven 
personal (sex, age, mantal status, ethnicity, education, income, number of children) and five employee 
characteristics (current Head Start position, years in current position, total years in Head Start, total years 
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working with children in non-Head Start positions, training on similar topics) analyzed. Analyses also 
indicated no significant within group differences on these same variables for both the training and 
comparison groups. Due to the lack of significant differences within and between groups related to 
attrition, primary analyses included only those trainees (91) and comparisons (41) with complete 
evaluation data (pretraining, 1 month and 6 months posttraining). In addition, secondary analyses were 
completed using subjects who had pretraining and 1 month posttraining data (136 training and 97 
comparison subject). 

Program . Using t-tests and chi-squares it was found that comparison and trainee subjects 
differed on two of the six program characteristics analyzed. Individuals in the comparison group came 
from programs that served a more urban population (87% of the comparisons; 63% of the training group). 
In addition, the comparisons' programs had more center-based slots (M = 282) than trainees' programs 
(M = 201). Other than these two differences, analyses indicated that trainees and comparisons came 
from similar programs in terms of the average number of children served (241 for trainees, 306 for 
comparisons); average number of home-based slots (41 for trainees; 38 for comparisons); number of 
sites (8 for both), and rate of compliance with Head Start performance standards. 

Implementation of training 

Nature of training Information collected during training years 1, 2, and 3 indicated that the 
NEHSTC was successful in capitalizing on the power of the teaching center model by including ample 
participatory training. The average percent of training time allocated to participatory activities was 37% 
with 23% didactic, 33% discussion, and 6% simulation. Table 2 lists the percent of time allocated to these 
four different learning activities for all training topics delivered. 

Quality of training Both trainee ratings and independent observations completed by the 
evaluation team suggested the training was high quality. Overall, trainees rated the content of training as 
relevant and useful, the trainers as knowledgeable and organized, and the “laboratory” nature of the 
participatory training as invaluable. The evaluation team provided high ratings for the overall quality of the 
various learning activities. On a 9-point scale, the mean ratings provided by the evaluation team were 
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7.90 for participatory activities, 8.19 for didactic, 8.03 for discussion, and 7.83 for role play and simulation 
activities. 

Outcomes: Impact of training 

Primary Analyses. Analysis of the outcomes for training years 1, 2, and 3 indicated that the 
NEHSTC training had a significant impact. A 2 (group) X 2 (time) repeated measures analysis of 
covariance, using the self ratings collected prior to training as the covariate, indicated a significant 
difference between the groups over time, Wilks' Lambda F (2, 1 28) = 1 2.23, p = .001 . Figure 1 shows the 
adjusted least square means. A Tukey's HSD indicated trainees reported significant gains in knowledge, 
skills, and expertise from pretraining (adjusted M = 53.39) to posttraining at 1 month (adjusted M = 59.10) 
and 6 months (adjusted M = 61 .15). The comparisons did not report significant gains from pretraining 
(adjusted M = 53.39) to posttraining at 1 month (adjusted M = 54.71) and 6 months (adjusted M = 53.63). 

A similar pattern was found in the analysis of supervisor ratings. A 2 (group) X 2 (time) repeated 
measures analysis of covariance, using the supervisor pretraining ratings as the covariate, indicated a 
significant difference between the groups over time, Wilks' Lambda F (2,127) = 14.62, p = .001. Figure 2 
shows the adjusted least square means for the supervisor ratings of trainees and comparisons. Again 
using a Tukey's HSD, supervisors reported significant gains for trainees from pretraining (adjusted M = 

51 .62) to posttraining at 1 month (adjusted M = 57.68) and 6 months (adjusted M = 60.74). The 
supervisors did not report significant gains for the comparisons from pretraining (adjusted M = 51 .60) to 
posttraining at 1 month (adjusted M = 55.24) and 6 months (adjusted M = 50.83). 

Secondary analyses. Secondary analyses of covariance, using only the 1 month data as the 
dependent variables, were conducted to investigate if attrition between the 1 and 6 month data collection 
points impacted the outcomes. With the self-ratings, a significant difference was found between trainee (n 
= 136; adjusted M = 61.3) and comparison (n = 97; adjusted M = 55.2) groups at 1 month, F (1,230) = 
26.99, p = .001 A similar significant difference was found for the supervisor ratings provided for trainee 
(adjusted M = 59.45) and comparison (adjusted M = 55.06) groups at 1 month, F (1 ,230) = 14.68, p = 

.001 . These significant differences were consistent with the results of the ANCOVAs that included both 
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the 1 and 6 month data. These consistencies suggest the significant differences found in the primary 
analyses were not related to attrition, including potential differential drop out as a function of subjects' 
perceived competence. 



Conclusion 

Overall, the results of this evaluation of the NEHSTC present a positive assessment based on 3 
years of experience in implementing the participatory model as delineated in the ACYF national 
demonstration project. At 1 and 6 months after training, trainees reported increased knowledge, skills and 
expertise while comparison group subjects did not report such gains. Importantly, this self-report finding 
was corroborated by independent reports of the trainees' supervisors and independent observations of 
their job performance 6 months after training. 

These findings clearly have implications for future training initiatives given the continuing 
emphasis on quality of Head Start services (U.S. Department of Health and Human Services, 1993). The 
positive findings, in terms of both training quality and impact, suggest that the Teaching Center 
participatory model has significant potential as a new approach to Head Start staff development. Given 
the recent restructuring of the training and technical assistance system in conjunction with the new Head 
Start Performance Standards, information concerning training formats that enhance quality are particularly 
relevant. The results reported in this study suggest that participatory training should be included in the 
menu of training options available within Head Start. Additionally, these findings are relevant for broader 
efforts to improve early care and education programs nationwide. 
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Table 1 

Outcome Evaluation Design 



Data Source 



Group 



Training Comparison 

Groups Group 



1. Pre-training data 

- trainee self-report x 

- supervisor report X 

2. Evaluation of training sessions 

- observation X 

- records review X 

- trainee report X 

3. Evaluation of training outcomes 

- trainee self-report 



1 month X 

6 months X 

- supervisor report 

1 month X 

6 months X 

- observation in trainee program 

6 months X 



X 

X 



X 

X 

X 

X 
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Table 2 

Percent of Time Allocated to Different Learning Activities for 93-96 Trainings 







Learning Activities 


Track 


Didactic 


Discussion Simulation Participatory 



Celebrating 


26 


49 


0 


25 


Diversity 


Food Safety 


26 


19 


15 


41 


Individualizing 


17 


24 


9 


50 


Services/Social Serv 


Individualizing 


19 


29 


6 


46 


Services/Education 


Child Initiated 


32 


32 


10 


27 


Curriculum 


Menu Planning 


22 


50 


3 


25 


Home Visitor 


22 


29 


2 


46 


Average Percent 


23 


33 


6 


37 



Note . Numbers in table are percentages of time allocated to different learning activities. 
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Table 3 



Means for Self and Su pervisor Ratings for Trainee and Comparison Subjects 



Ratings 


Group 




Self 




SuDervisor 




pre 


1 month 


6 months 


pre 


1 month 


6 months 


Trainee 


51.66 


57.99 


60.73 


50.56 


56.95 


60.46 


n = 91 


(9.73) 


(9.17) 


(9.27) 


(8.92) 


(10.03) 


(9.86) 


Comparison 


57.23 


57.19 


54.56 


54.64 


57.23 


51.45 


n = 41 


(10.47) 


(11.80) 


(9.21) 


(14.38) 


(12.82) 


(9.95) 



Note. Standard deviations are reported in parentheses. 



SELF-RATING MEANS 



Figure 1 

Ad j usted Means for Self-Rati n gs of Trainee and Comparison Groups 
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SUPERVISOR MEANS 



Figure 2 
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